U.S. Pat. No. 6,629,220 describes dynamic arbitration of memory access requests. A circuit with a plurality of queues of memory access requests is described, associated for example with various processors, I/O interfaces hardware accelerators etc. An arbiter transfers the requests from these queues to a global queue. When doing so, the arbiter has to select the order in which it transfers the requests from different queues. This is done by assigning relative priorities to the different queues and by transferring each time the first pending request from the queue with highest priority.
Unfortunately, this means that requests from queues with low priority may have to wait longer dependent on the number of pending requests in other queues with higher priority. Moreover, the delays may even be influenced by lower priority requests, once the arbiter has committed itself to handle such a lower priority request. Waiting for handling of requests from other queues can result in long and unpredictable knock-on delays in processors that issued requests to the low priority queues. Such knock on delays can be reduced by giving higher priority to requests when the processor has to wait for a response. Thus, for example memory read requests can be given a higher priority than memory write requests, provided of course that they concern different memory addresses.
U.S. Pat. No. 6,629,220 describes that an instruction can be used to give one queue priority over the others, and that queues with certain types of request, such as requests for use in synchronous communication, may be given higher priority than other queues with other types of requests, such requests for use in asynchronous communication.
Extremely long delays can be avoided by dynamically changing the relative priority of different queues. Generally, priority changes may be based on the duration for which a request has been queued, the number of requests that have been serviced from different queues, bus activity etc.
Although measures of this type help to reduce delays, a processor may still have to wait a long and unpredictable time interval before it gets a response to a request.